Derive exam markers from what the kursplan actually examines - #64
Merged
Conversation
DD1327 carried an exam marker in every cohort but HT2022, and it has no exam: its
kursplan examines HEM1 (Individuella hemuppgifter) and PRO1 (Individuellt
projekt). HT2022 was right only by accident, because it reads an older kursplan
where the same coursework was coded IND1. Two bugs produced that, and both had
wider effects than the one course.
'HEM' WAS NOT A TENTAMEN PREFIX
EXAM_MODULE_PREFIXES was ['TEN', 'HEM'], and its comment recorded the reasoning:
"HEM = hemtentamen (DD1327 is HEM1+PRO1 and the curated data does mark an exam
for it)". The rule was derived from DD1327's curated value, which was itself
wrong, and then propagated that value to every other cohort - a loop where our
output justified the rule that produced it.
Measured over all 278 courses in the data: all 48 HEM* modules are titled
"hemuppgift" or "hemuppgifter", homework, and not one is a tentamen. The only
genuine take-home exam anywhere in the dataset is coded EXA1 ("Hemtentamen"),
which a HEM prefix misses entirely.
The test is now: code starts with TEN, or the module title says tentamen. The
code half keeps the few TEN modules with an unusual title (TEN1 "Examination",
TEN1 "Skriftlig test", TEN1 "Kontrollskrivning"); the title half picks up EXA2
"Hemtentamen". TEN* is 569 modules, titled tentamen / skriftlig tentamen /
problemtentamen.
THE MODULE PARSER READ ONLY THE FIRST MODULE
The page concatenates every module of a kursplan into one string with no
separator. PAGE_MODULE_RE ended in a greedy ([^|<]+) for the grading scale, which
swallowed the rest of the string - so it parsed 933 modules where a correct split
finds 2275, truncating 66% of all kursplan versions. examBearing.length could
therefore never exceed 1, which silently defeated the "one exam per exam-bearing
module" rule that examsForPeriods documents. Fixing the parser makes the code do
what its own comment already claimed.
Two further details the old pattern got wrong: a module code can end in a letter
(SF2930's TENA - Skriftlig tentamen is a real exam that was never matched), and
Swedish initials matter (ÖVN1 was matched from the V, giving 61 phantom "VN"
prefixes).
MEASURED AGAINST THE CURATED FILES
Agreement with the hand-authored CTFYS exams goes 16/26 -> 23/26; single-period
courses 12/15 -> 14/15 and multi-period 4/11 -> 9/11. The one remaining
single-period miss is DD1327 itself, where the curated value is the error, so that
is effectively 15/15 - which matches the claim that a single-period course's exam
sits in its teaching period.
The two remaining multi-period misses are the known-hard case rather than a new
one. SE1055 has one tentamen and the curated file puts it in the last of its two
periods while our convention picks the credit-majority period; SG1112 genuinely
has two tentamina (TEN2 Problemtentamen, TEN1 Teoritentamen) where the curated
file records one. Neither is reachable without real timetable data.
THE CURATED FILES NEEDED CORRECTING TOO, AND --exams DOES IT
Auditing every curated 'exams' array against the kursplan in force for the current
läsår: CTFYS agreed on 25 of 26, but CINEK on only 17 of 50 - 33 of its courses
record no exam while their kursplan carries an explicit TEN module, and SG1109
carries two. CTMAT was missing 5 and CFATE 1. Those are unpopulated fields rather
than considered judgements.
The new --exams flag reconciles a curated file, deliberately in three different
ways:
- no exam recorded but the kursplan has a tentamen -> filled
- an exam recorded but the kursplan has none -> cleared (DD1327)
- both present but the period differs -> REPORTED, curated kept
The third rule is the important one. A coordinator's placement outranks our
highest-credit convention, which is measured at roughly half on multi-period
courses, so SE1055 and SG1112 keep their hand-authored values and are printed for
a human to settle. This mirrors --prereqs, where a curated reading of the same
free text also wins.
WHAT ACTUALLY CHANGED
Cohort files, courses carrying an exam marker (HT2026 shown, the other cohorts
move with it): CTFYS 15->21, CTMAT 19->22, CFATE 10->25, COPEN 4->7,
CINEK 15->32, CMAST 14->29, CMATD 14->22, TIEMM 36->71. In total 127->229. Most
of that is the parser fix making previously invisible TEN modules visible.
Curated files: CINEK 0->33 - it recorded no exams at all - CTMAT 17->21,
CFATE 18->19, CTFYS 22->21, COPEN unchanged. Eight disagreements about which
period were reported and left alone (SG1112, SE1055, SG1132, SF1668, SF1682,
SD1120, SG1133).
SA1006 was the second wrong curated value, of the same class as DD1327. It
carried a by-year map, {"Year1":["P2"],"Year2":["P4"],"Year3":["P2"]}, while its
kursplan examines PRO1 through PRO5 - five projects, all P/F, no exam at all.
Cleared.
ORAL EXAMS ARE COUNTED BUT FLAGGED
A "Muntlig tentamen" is a tentamen and is counted, but an oral exam is usually
booked individually rather than sitting in the scheduled examination period,
which is what the marker on the chart means. Only three exist in the whole data
set - ME2322, ME2323 and MJ1141 - too few to invent a rule from, so they are
counted and reported for a coordinator to overrule.
Also: flag() now dedupes. Exam derivation runs twice per course, once during
enrichment and once after the cohort's kursplan version is chosen, so identical
review lines were being printed twice.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
DD1327 carried an exam marker in every cohort except HT2022 — and it has no
exam. Its kursplan examines
HEM1(Individuella hemuppgifter) andPRO1(Individuellt projekt). HT2022 was right only by accident: it reads an older
kursplan where the same coursework was coded
IND1.Two bugs produced that, and both reached well beyond the one course.
1.
HEMwas not a tentamen prefixEXAM_MODULE_PREFIXESwas['TEN', 'HEM'], and its own comment recorded why:The rule was derived from DD1327's curated value — which was itself the error —
and then propagated that value to every other cohort. A loop where our output
justified the rule that produced it.
Measured over all 278 courses in the data:
TENHEMEXANot one
HEM*module is a tentamen, and the only genuine take-home exam in thewhole dataset is coded
EXA1, which aHEMprefix misses entirely.The test is now code starts with
TEN, or the module title says tentamen. Thecode half keeps the few
TENmodules with an unusual title (TEN1 "Examination",TEN1 "Skriftlig test",TEN1 "Kontrollskrivning"); the title half picks upEXA2 "Hemtentamen". Neither test alone is sufficient — I checked.2. The module parser read only the first module
The page concatenates every module of a kursplan into one string with no
separator.
PAGE_MODULE_REended in a greedy([^|<]+)for the grading scale,which swallowed the rest of the string:
It parsed 933 modules where a correct split finds 2275, truncating 66% of
all kursplan versions. So
examBearing.lengthcould never exceed 1, silentlydefeating the "one exam per exam-bearing module" rule that
examsForPeriodsdocuments. Fixing the parser makes the code do what its own comment already
claimed.
Two more details the old pattern got wrong: a module code can end in a letter
(SF2930's
TENA - Skriftlig tentamenis a real exam that was never matched), andSwedish initials matter (
ÖVN1was matched from theV, giving 61 phantomVNprefixes).
Measured against the curated files
Agreement with the hand-authored CTFYS exams:
The remaining single-period miss is DD1327 itself, where the curated value is the
error — so effectively 15/15, which is what the riktlinje implies: a
single-period course's exam sits in its teaching period.
The two remaining multi-period misses are the known-hard case, not a new one.
SE1055 has one tentamen and the curated file puts it in the last of its two
periods while our convention picks the credit-majority one; SG1112 genuinely has
two tentamina (
TEN2 Problemtentamen,TEN1 Teoritentamen) where the curatedfile records one. Neither is reachable without real timetable data.
The curated files needed correcting too
You signed off on this, and here is the audit it rests on — every curated
examsarray against the kursplan in force for the current läsår:
TENmoduleCINEK's 33 are unpopulated fields, not considered judgements — SG1109 carries a
Problemtentamen and a Teoritentamen and records neither.
--examsreconciles them, in three deliberately different waysThe third rule is the important one. A coordinator's placement outranks our
highest-credit convention, which is measured at roughly half on multi-period
courses — so SE1055 and SG1112 keep their hand-authored values and are printed
with a link for a human to settle. This mirrors
--prereqs, where a curatedreading of the same free text also wins.
What actually changed
Courses carrying an exam marker, HT2026 (the other cohorts move with it):
Most of that is the parser fix making previously invisible
TENmodules visible.It is a large, visible change to the chart — worth a look before merging.
Curated files: CINEK 0 → 33 (it recorded no exams at all), CTMAT 17 → 21,
CFATE 18 → 19, CTFYS 22 → 21, COPEN unchanged. Eight period disagreements were
reported and left alone: SG1112, SE1055, SG1132, SF1668, SF1682, SD1120, SG1133.
SA1006 was a second wrong curated value, same class as DD1327. It carried a
by-year map —
{"Year1":["P2"],"Year2":["P4"],"Year3":["P2"]}— while itskursplan examines
PRO1–PRO5: five projects, all P/F, no exam at all. Cleared.One judgement call left open
A "Muntlig tentamen" is a tentamen and is counted — but an oral exam is usually
booked individually rather than sitting in the scheduled examination period, which
is what the marker on the chart means. Only three exist in the whole dataset
(ME2322, ME2323, MJ1141), too few to invent a rule from, so they are counted and
flagged in the review output for you to overrule.
Verification
validate-data --cohorts0 errors, 225 warnings (unchanged) ·lintandtsc --noEmitclean · all 40 cohort files regenerated and re-aligned ·prerequisite-review/*.mdcame out byte-identical, confirming this touches onlyexams.Also:
flag()now dedupes. Exam derivation runs twice per course — once duringenrichment, once after the cohort's kursplan version is chosen — so identical
review lines were printed twice.